Rendering methods for augmented reality

Augmented reality (AR) has become a promising and fast-growing application of computer graphics over the course of the last years. Augmented reality systems overlay computergenerated graphical information over the view of the real world. Several main research challenges can be identified in the field of augmented reality. These are the design of advanced display devices (e.g., head-mounted displays), camera tracking, system design, user interaction, and rendering. While a major part of the previous work focused on the problems of system design, camera tracking, and applications of AR, this thesis puts a different emphasis on the relatively underrepresented aspect of rendering techniques. In this thesis, several novel methods for displaying augmented video streams are explored. In the first part of this thesis in Chapter 2, the design and implementation of a novel system for medical augmented reality are discussed [22]. The ARGUS framework is a new augmented reality system based on a commercial surgical navigation device. Since it does not require any additional hardware components, a transition into the clinical practice can be facilitated. Several extensions of the basic framework are described, including a hybrid tracking scheme, a user interaction library, and a method for handling occlusion. The latter algorithm makes it possible to correctly render the occlusion of graphical objects by the anatomy of the patient, leading to a more realistic and easily comprehensible output. This approach is one of the advanced rendering methods for augmented reality investigated in the context of this thesis. The second part of this thesis, Chapter 3, introduces the concept of stylized augmented reality, which applies artistic or illustrative stylization methods to augmented reality video streams [15]. Since the same type of stylization is applied to virtual and real scene elements, they become difficult to distinguish. This way, a novel augmented reality experience is created, and possibly even a better immersion. Real-time algorithms for cartoon-like and painterly brush stroke stylization of augmented video streams are described. The exploitation of programmable graphics hardware for this purpose is discussed. Moreover, the results of a psychophysical study on the discernability of virtual objects in stylized augmented reality are presented. At the beginning of Chapter 4, which is the third part of this thesis, a novel illustrative visualization method is described. This new rendering algorithm for iso-surfaces and polygonal models generates an illustrative representation of a surface and structures hidden behind it [12]. The method is designed for the programmable rendering pipelines of modern graphics hardware and is capable of displaying complex models in real-time. An extension of this newly developed illustrative display style was also applied to augmented reality video streams. This system constitutes another realization of the concept of stylized augmented reality.